Adaptive simulation rendering on sensory load

ABSTRACT

An apparatus is disclosed for conditioning and adapting an individual to real-world environments through computer simulations. The subject is fully or partially immersed in a computer-simulated environment for a time-limited session. Real-time monitoring of the human subject is performed for a change in a sensor-derived, quantified sensory load level. Responsive to an increase in sensory load level, the computer-simulated environment modulates the sensory complexity of one or more features of the simulation wherein the human subject adapts to increasingly complex environments.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S.Non-Provisional patent application Ser. No. 16,997,404, entitled“Physiologic Responsive Rendering of Computer Simulation”, filed on Aug.19, 2020, which claims priority to U.S. Non-Provisional patentapplication Ser. No. 16/801,333 (now U.S. Pat. No. 10,783,800), entitled“Sensor-Based Complexity Modulation for TherapeuticComputer-Simulations”, filed on Feb. 26, 2020, the contents of which areherein incorporated by reference.

BACKGROUND OF THE INVENTION 1. Field of the Invention

This invention relates to adapting the output of immersive computersimulations for individuals responsive to physiologic data indicative ofthe sensory load the computer simulation has on the individual.

2. Brief Description of the Related Art

The Center for Disease Control currently reports the rise in the numberof students with Autism Spectrum Disorder (ASD) to be of epidemicproportions. Despite numerous curricula being developed to supportindividuals with ASD, none employs a virtual environment context thatcan provide skills in EF using approximations of practice aligned withthe individual's real life. Despite the increased diagnosis of ASD,existing technologies are not well-aligned to provide a safe, nimble,scaffolded stimulation, and personalized practice space for those withASD to master critical skills.

Research and inventions for individuals with ASD learningsocial/emotional and communication skills (EF) through a virtualenvironment does not currently exist. Although “human-to-human-based”training has value for those with ASD, this methodology has been shownin practice to be not effective and to create anxiety. Research in videomodeling has been proven effective, and this work enhances orsupplements past success in using video models in an immersive virtualenvironment.

ASD is just one example of a condition that is exacerbated by sensoryoverload. Sensory overload has been found to be associated with otherdisorders and conditions such as schizophrenia, Tourette syndrome,attention deficit hyperactivity disorder (ADHD), obsessive-compulsivedisorder (OCD), attention deficit disorder (ADD), emotional behavioraldisorders (EBD), and post-traumatic stress disorder (PTSD). Generalsymptoms include irritability; over-sensitivity to movement, sights,sounds or touch; difficulty with social interactions; muscle tension;outbursts; difficulty making eye contact; overexcitement; repeatedlychanging activities without completing tasks; hyperhidrosis; andrestlessness.

Sensory overload is often addressed by participation in occupationaltherapy, outright avoidance, or limiting of environmental stimuli.Unfortunately, common environmental elements are nearly impossible toevade, including mass media, urbanization, crowding, digital immersiontechnologies, and ambient noise. For many young individuals, students,adults, and military veterans suffering from conditions related tosensory overload, limiting interactions with sources of impactingstimuli may not be an option when furthering educational, career andsocial objectives. What is needed is a way to adapt a simulatedenvironment to the sensory thresholds of each individual to developexecutive functions without overstimulation of the individual orunder-challenging the individual's capabilities.

BRIEF SUMMARY OF THE INVENTION

The long-standing but heretofore unfulfilled need for EF development isnow met by a new, useful, and nonobvious invention. This interventionreplicates interactions in a controlled environment found to transfer EFskills back into the home, school, and community life of individualshaving conditions such as ASD, PTSD, and ADD to name a few. Specificobjectives are set in the environment with computer-automated avatarsinteracting in a controlled setting to work on the development of EFskills. For work in EF this allows for controlled and repeated practice,which has been found to impact communication skills of students withASD.

With the report by the CDC that 1 in 59 children are on the ASDspectrum, this complex yet adaptable system provides a safe andeffective tool to work on targeted skills in EF. This virtualenvironment allows for students with sensory overload, including ASD whomay be anxious about a situation to experience the situation and work onEF prior to the real-world situation (e.g., doctor's visit, asking out apeer on a date, asking for help in a situation.).

Throughout a series of treatments for a student, the scenarios arecustomized through visual access to the students with sensory overload,including ASD and by the AI using machine learning techniques to selectbehaviors based on a variety of observations including facialexpressions, body poses, vocalizations, and verbalizations. Beyond thecustomized setting, the system can personalize the level of verbalinteraction and social-emotional situations experienced to provide forrepeated practice of EF skills. The intensity of the scenario can beescalated or de-escalated depending on the objective, tolerance, and thedesired outcome specified by the intervention team aligned with learningversus mastering an EF skill. Sensitive topics such as sexual assault,personal hygiene, or dating can be addressed along with more simplistictopics such as talking with a peer in a group to basic communicationskills for employment or self-sufficiency.

When a student with sensory overload, including ASD, first steps intothe simulator, familiarization with the selected environment isestablished. The targeted EF can be practiced over and over again untila desired level of mastery is reached and sustained. Target areas thatare not met can be re-adjusted unlike in a traditional game-basedsituation that requires writing new code or role-play that results insubject fatigue. This virtual system can achieve a scaffolded level ofresponse to obtaining EF skills.

An embodiment of the invention includes an apparatus for a therapeutictreatment of a human subject with environmental anxiety disorder. Theapparatus generally consists of sensors, video displays, computerprocessor, and memory storage devices. A control module comprising acomputer processor is communicatively coupled to a simulation datastore, the simulation data store has machine-readable values forcomputer-generated features in a computer-simulated environment in whichthe human subject is immersed and tasked to perform an executivefunction. The executive function for a child could be simply stackingblocks. An executive function for an autistic adult could be cooking asimple meal in a kitchen. A common characteristic of the executivefunction assigned is that a sufficient increase in anxiety state by thehuman subject hinders successful completion of the executive function.For example, a child may become anxious when surrounded by two or moreadults, subject to loud ambient noise and/or when immersed into aharshly lit environment. All these features of the real world thatimpact the child's ability to function and perform common tasks can besimulated and adjusted dynamically in a computer-simulated environment.

In this embodiment of the invention, the computer-generated featuresinclude visual objects and an audio output. The visual objects could betwo-dimensional or three-dimensional, and the audio output could besingle or multi-channel (e.g., spatial “surround sound”). A renderingmodule is communicatively coupled to the control module. The renderingmodule is further communicatively coupled to a graphic processing unit(GPU) that generates the visual objects in the computer-simulatedenvironment through a visual display device.

The visual display device is communicatively coupled to the renderingmodule and the GPU, the visual display device displaying the visualobjects in the computer-simulated environment. The visual display devicemay include single panel display monitors, multi-panel display monitors,rear projection displays, front projection displays, head-mountedvirtual reality displays, and head-mounted augmented reality displays.An audio processing unit (APU) is communicatively coupled to therendering module, and the APU generates the audio output in thecomputer-simulated environment in single or multiple audio channels.

An array of sensory variables is accessible by the rendering module, thesensory variables quantify an amount of visual and audible informationgenerated by the rendering module and presented in thecomputer-simulated environment. The sensory variables may includeaudible noise, audio volume, quantity of visual objects in theenvironment, movement of visual objects in the environment, polygoncount of rendered objects in the environment, lighting complexity of theenvironment, texture complexity of rendered objects in the environment,and rendered frames-per-second.

Communicatively coupled to the control module is a sensing module.Further communicatively coupled to the sensing module are at least oneor more digital sensors, including cameras, radar, thermometers, heartrate monitor, pulse-oximeters and microphones. The sensing modulereceives non-transitory data signals from the digital sensors indicativeof a physiological parameter of the human subject. From data receivedfrom the sensing module, a real-time anxiety value of the human subjectis derived from the data signals indicative of the physiologicalparameter, the real-time anxiety value readable by the control module.By way of example, a common physiological parameter may be heart rate.This can be read by a pulse meter or pulse-oximeter directly affixed tothe human subject. Alternatively, it may be read remotely by radar,ultrasonic sensor or other means known in the art of pulse rateacquisition.

An anxiety threshold datastore is communicatively coupled to the controlmodule, the anxiety threshold datastore stores an upper anxiety statevalue constant representing a physiological diminished capability ofperforming executive functions and a lower anxiety state value constantassociated with a sufficiently low physiological anxiety state wherebyexecutive functions may be successfully performed with additionalstress-induced anxiety, the upper anxiety state value and the loweranxiety state value computed by one or more quantitative factorsselected from the group consisting of pulse rate, respiration rate, skintemperature, and diaphoresis.

An anxiety threshold function is operable on the control module, theanxiety threshold function receiving the real-time anxiety value of thehuman subject, the upper anxiety value constant and the lower anxietyvalue constant whereby the anxiety threshold function returns a lowresult responsive to the real-time anxiety value of the human subjectbeing less than the lower anxiety value; a high result responsive to thereal-time anxiety value of the human subject being greater than theupper anxiety value; and an inbounds result responsive to the real-timeanxiety value of the human subject being above the lower anxiety valueand less than the upper anxiety value.

Responsive to a low result returned from the anxiety threshold function,the control module instructs the rendering module to increase the valuesof the sensory variables to thereby increase the amount of visual andaudible information generated (e.g., simulation complexity) by therendering module and presented within the computer-simulatedenvironment. Responsive to a high result returned from the anxietythreshold function, the control module instructs the rendering module todecrease the values of the sensory variables to thereby decrease theamount of visual and audible information generated by the renderingmodule and presented within the computer-simulated environment. Finally,responsive to an inbounds result returned from the anxiety thresholdfunction, the control module instructs the rendering module to maintainsubstantially the same values of the sensory variables to therebysustain the same amount of visual and audible information generated bythe rendering module and presented within the computer-simulatedenvironment.

The human subject therapeutically develops proficiency in the executivefunction through the use of the apparatus by optimizing the amount ofvisual and audible information presented within the computer-simulatedenvironment to sufficiently challenge the human subject by increasingvisual and audible information rendered in the computer-simulatedenvironment without detrimentally overloading the human subject withexcessive visual and audible information.

An embodiment of the invention includes a method of developing executivefunctions for a human subject having an anxiety disorder in acomputer-simulated environment. The method includes establishing abaseline anxiety level for the human subject wherein the baseline isassessed by automatically monitoring the phenotypic anxiety level of thehuman subject by one or more computer coupled sensors. The baseline maybe obtained prior to the start of the computer simulation or within acomputer simulation that is relatively “idle” without significantinteraction or tasks assigned to the human subject. Once the baselineanxiety level is obtained, the human subject is tasked with an executivefunction wherein the human subject is fully or partially immersed in thecomputer-simulated environment for a time-limited session.

Executive function is often considered an umbrella term for theneurologically-based skills involving mental control andself-regulation. Executive functions include task initiation, impulsecontrol, self-monitoring, emotional control, flexible thinking, workingmemory, planning, and organization. Accordingly, for a young child,stacking of blocks may involve several executive functions that may beimpacted by environmental stimuli. In the case of a computer-simulatedenvironment, the child (the human subject) may have trouble performingthis task during preschool with other children, teachers, ambientnoises, lighting, smells, and visual complexity. Accordingly, for thatchild, a virtual schoolroom may be the computer-simulated environmentwherein the intensity of the sensory experience may be controlled.

In yet another example, a military veteran who has combat-induced PTSDmay have trouble performing executive functions related to amanufacturing job wherein sounds, vibrations and visuals may causeanxiety that detracts from her ability to perform her job. In such acase, the executive function task monitored during thecomputer-simulated environment may be assembling a wiring harness for arobotic arm. As needed, the tasks may be generic but related to specificobjectives such as manual dexterity, memory retention, organization, andprioritization. Alternatively, they may also be specific to certainoccupations, educational objectives, or even competitive sports.

In yet another example, a young professional soccer player exhibits ahigh level of play during practice but suffers during matches in largestadiums. The executive function in this scenario may be his ability tocontrol the ball, and the computer-simulated environment may be that ofa large stadium during a World Cup match. His anxiety level may beimpacted by the ambient noise of the stadium or even responsive to fansfrom the opposing team heckling him. In such an application, the subjectmay be immersed in an augmented reality environment wherein the stadiumvisuals and noise exist in combination with a trainer engaged in drillswith the subject In the soccer embodiment (and applicable to others),the frame rate of the rendered simulation may be adjusted. Particularlyfor simulations with fast-moving elements, the frame rate may beincreased from 30 to 60 to even 120 frames per second or above (subjectto the limitations of the display device). For example, MICROSOFTCORPORATION publishes target frame rates of his HOLOLEN product at 60frames per second. For hardware branded WINDOWS MIXED REALITY PCs thetarget frame rate is 60 frames per second. For hardware branded WINDOWSMIXED REALITY ULTRA PCs, the target framerate is 90 frames per second.

Additional computer-generated environments may include, but are notlimited to, battlefield simulations for military personnel training,classroom environments for students and instructors, vehicle simulationsfor driving and flight training, aircraft simulations for flight anxietytherapy, hospital simulations for training for medical personnel andworkplace simulations for career and vocational training. It isimportant to note that the physiological monitoring and modulation ofthe computer-simulated environmental complexity of this invention doesnot make the executive function more difficult or easier (e.g.,modifying the difficulty level of a game). Rather, the apparatusmodifies the sensory complexity of the computer simulation. It is theanxiety-reactive stimulation modulation of an individual undergoingtherapy that enables prolonged sessions and engagement by the individualin an increasingly complex and realistic environment. The digitalsensors react more consistently and quickly than is possible by humanobservation to acclimate the computer-simulated environment to thecapabilities and tolerance of the treated individual.

The human visual system can process between 10 and 12 images per secondand perceive them individually, while higher rates are perceived asmotion. Higher frame rates for fast-moving computer-simulatedenvironments produce a more intense sensory experience. Therefore,responsive to sensor-detected elevated anxiety levels, frame rates maybe reduced in the simulation.

In yet another application of the invention, an individual sufferingfrom flying anxiety may be immersed in a virtual, augmented, or mixedreality environment simulating the cabin of a commercial aircraft In anembodiment of the invention, the computer simulation may also includevibrations and tilting of the subject's chair to simulate takeoffs,landings, and turbulence. The computer simulation can modify theintensity of the simulation by controlling the number of passengers(typically automated avatars), the behaviors of the passengers (e.g.,belligerent passengers, crying children, sleeping passengers), ambientnoise levels, dialog loops of the automated avatars, and the like. Thesubject's phenotypic anxiety level (e.g., pulse, temperature, facialexpressions, etc . . . ) automatically modifies the simulation within athreshold that maintains a therapeutic benefit to the subject withoutoverwhelming the subject. The phenotypical anxiety level detected bysensors may anticipate the conscious anxiety of the subject so that thecomputer simulation reacts faster than a subject or human observer couldunder manual control.

The human subject is automatically monitored in real-time for a changein phenotypic anxiety level by the one or more computer coupled sensors.The sensor may include cameras to monitor facial expressions, bodymovement, and eye movement An infrared, remote thermometer may measurebody temperature. A pulse-oximeter may measure pulse rate. A microphonemeasures speech patterns and the volume in which the human subjectspeaks.

Responsive to an increase in the sensor-detected anxiety level, acomputer coupled to the sensor and coupled as well to thecomputer-simulated environment decreases the sensory complexity of oneor more features of the computer-simulated environment. The features ofthe computer-simulated environment may include audible noise, audiovolume, quantity of visual objects in the environment, movement ofvisual objects in the environment, polygon count of rendered objects inthe environment, lighting complexity of rendered objects in theenvironment, texture complexity of rendered objects in the environment,olfactory dispersions, tactile feedback frequency to the human subject,tactile feedback intensity to the human subject, frames per secondrendered and simulation event repetition. Responsive to a decrease inthe sensor-detected anxiety level, the computer automaticallyreintroduces sensory complexity to the computer-simulated environmentwherein the human subject therapeutically develops proficiency inexecutive functions in increasingly complex environments.

Where a plurality of phenotypic anxiety levels are obtained through thesensors, they may each be normalized to a proportion of overall subjectanxiety and aggregated to a total anxiety level value wherein a baselineanxiety level is dynamically linked to a median level ofcomputer-simulated environment complexity. For example, a rise in pulserate by 20 beats per minute may be weighted more heavily that a 4decibel rise in the subject's speech volume.

A threshold total anxiety level value may be calculated. When such valueis reached the computer automatically reduces computer-simulatedenvironment complexity. The threshold total anxiety level may be higheror lower than the baseline wherein simulation complexity does notimmediately increase or decrease until the threshold value is reached.For example, if a baseline anxiety level of 50 is calculated prior tothe start of the simulation, increases in simulation complexity do notinitiate until the threshold falls under a value of 40, nor do decreasesin simulation complexity or until the threshold anxiety level of 60 isreached.

The computer-simulated environmental complexity applied by the computermay be inversely related to the sensor-detected total anxiety levelwhereby lower total anxiety levels cause the computer to increasecomputer-simulated environmental complexity and higher total anxietylevels cause the computer to decrease computer-simulated environmentalcomplexity. The computer-simulated environment may be initiated at a lowcomplexity upon initiation of the session and incrementally increasedover the session until a threshold anxiety level is reached, at whichtime the computer either maintains or reduces the complexity of thecomputer-simulated environment.

In an embodiment of the invention, an executive function evaluationapparatus is introduced providing real-time quantitative values on thecompetency of the patient performing the executive function wherein thecomputer automatically adjusts the complexity of the computer-simulatedenvironment responsive to the executive function competency (exclusivelyor in association with the sensor data) wherein higher levels ofdetected competency increase complexity of the computer-simulatedenvironment and lower levels of detected competency decrease thecomplexity of the computer-simulated environment. The executive functionevaluation apparatus may measure dexterity with physical objects. Thiscould be as simple as detecting through mechanical means involving theinsertion of a round peg into a round hole or a cube into a squareopening. The apparatus may be more complex measuring the dexterity ofthe human subject with a wireless device containing accelerometers thatcommunicate orientation responsive to directives of thecomputer-simulated environment (e.g., point to an object).

Alternatively, the executive function evaluation apparatus measurescomprehension of the information presented in the computer simulatedenvironment by the movement of physical objects such as an ordering orarranging tasks. In yet another embodiment of the invention, theexecutive function evaluation apparatus measures comprehension byentries on a computing device responsive to prompts presented in thecomputer-simulated environment. The computing device may be a keyboard,touchscreen or one detecting gestures via motion-capture.

In an embodiment of the invention, the computer-simulated environmenthas at least one avatar. The avatar may be automated, responding topredefined actions by the human subject. In yet another embodiment ofthe invention, a plurality of avatars may be presented in thecomputer-simulated environment. Additionally, automated avatars mayserve as “extras” in the simulation to incrementally increase simulationcomplexity. An embodiment of the invention automatically controlscertain aspects of the avatar to challenge the subject or lessen anxietyon the subject as determined by the sensor-detected anxiety level of thesubject. For example, the aggressiveness of the posture and poses anavatar takes may increase anxiety but may also be useful in training asubject for real-world experiences. The avatar's facial expressions,voice pitch, timbre, and volume may all be automatically or partiallycontrolled responsive to the anxiety state of the subject undergoingtherapy. For example, a subject interacting with an “anti-bully”simulation may learn to control their response to anxiety-pronesituations which is monitored by their pulse, body temperature and othersensor-detectable biometrics. The avatar could also be an animal. Forexample, an individual that suffers from anxiety responsive tointeractions with dogs might find the avatar representation of a dogcalms in direct relation to the subject's ability to low her anxietylevel determined by her sensor-detected pulse and facial expressions.Specifically, the dog has several “micro-poses” that show differentstates of emotion including friendly, scared, angry and relaxed.

In addition to a method, the present invention includes an apparatusembodiment which incorporates a computing device, a plurality of sensorscoupled to the computing device, the sensors establishing a baselineanxiety level for the human subject, and the baseline assessed byautomatically monitoring the phenotypic anxiety level of the humansubject. A projection device is communicatively coupled to the computingdevice; the projection device displays the computer-simulatedenvironment to the human subject. The projection device may includesingle panel display monitors, multi-panel display monitors, rearprojection displays, front projection displays, head-mounted virtualreality displays, and head-mounted augmented reality displays.

An executive function monitoring apparatus is communicatively coupled tothe computing device, and the monitoring apparatus quantifies thecompetency of the human subject to perform one or more executivefunctions while immersed in the computer-simulated environment. Anavatar module is communicatively coupled to the computing device; theavatar module causes an avatar to appear within the computer-simulatedenvironment and interact with the human subject. An anxiety level moduleis communicatively coupled to the computing device; the anxiety levelmodule automatically monitors, in real-time, the human subject for achange in anxiety level by the one or more computer coupled sensors. Asimulation complexity module is communicatively coupled to the computingdevice; the simulation complexity module responds to an increase in thesensor-detected anxiety level by decreasing the sensory complexity ofone or more features of the computer-simulated environment. Similarly,the simulation complexity module responds to a decrease in thesensor-detected anxiety level by automatically reintroducing sensorycomplexity to the computer-simulated environment, wherein the humansubject therapeutically develops proficiency in executive functions inincreasingly complex environments.

BRIEF DESCRIPTION OF THE DRAWINGS

For a fuller understanding of the invention, reference should be made tothe following detailed description, taken in connection with theaccompanying drawings, in which:

FIG. 1 is a diagrammatic view of an embodiment of the inventioncomprising computer components and software modules havingcomputer-readable instructions stored in computer-readable memoryexecuted by a computer processor.

FIG. 2 is a diagrammatic view of an embodiment of the invention whereina sensor-detected high pulse rate input value causes the computerprocessor to execute a function stored in computer memory to decreasepolygon counts rendered to a display device thereby lowering subjectanxiety.

FIG. 3 is a diagrammatic view of an embodiment of the invention whereina sensor-detected pulse rate input value is within thresholds causingthe computer processor to maintain the current level of simulationcomplexity (with regard to polygon count).

FIG. 4 is a diagrammatic view of an embodiment of the invention whereina sensor-detected pulse rate input value is within below thresholdscausing the computer processor to increase the current level ofsimulation complexity by reintroducing or increasing polygon countsthereby potentially increased human subject anxiety in a controlledmanner.

FIG. 5 is a diagrammatic view of an embodiment of the invention whereinmultiple sensor values (pulse and respiration) are inputted into theprocessor-executed function to return a directive to reduce simulationcomplexity, namely lowering background sound and frames-per-secondrendered.

FIG. 6 is a diagrammatic view of sensor monitoring of a subject foranxiety.

FIG. 7 is a graphics user interface view of sensor monitoring of asubject over time according to an embodiment of the invention.

FIG. 8 is a graphics user interface view of an audio control dialog forthe simulation output according to an embodiment of the invention.

FIG. 9 is a graphics user interface view of an environmental controldialog for the simulation according to an embodiment of the invention.

FIG. 10 is a graphics user interface view of a character control dialogfor the simulation according to an embodiment of the invention showing afirst avatar.

FIG. 11 is a graphics user interface view of a character control dialogfor the simulation according to an embodiment of the invention showing asecond avatar.

FIG. 12 is a graphics user interface view of a simulation rendering ofan avatar in a kitchen environment according to an embodiment of theinvention.

FIG. 13 is a graphics user interface view of a simulation rendering ofan avatar in a kitchen environment with more complex lighting andtexturing according to an embodiment of the invention.

FIG. 14 is a graphics user interface view of a simulation rendering of areduced-sized avatar in a kitchen environment according to an embodimentof the invention.

FIG. 15 is a graphics user interface view of a photo-realistic avatar ina classroom environment according to an embodiment of the invention.

FIG. 16 is a graphics user interface view of a simplified avatar withoutUV mapping in a classroom environment according to an embodiment of theinvention.

FIG. 17 is a graphics user interface view of a camera view of a humansubject with facial tracking points superimposed according to anembodiment of the invention.

FIG. 18 is a graphics user interface view of a haptic (tactile) controldialog according to an embodiment of the invention.

FIG. 19 is a graphics user interface view of an olfactory sense controldialog according to an embodiment of the invention.

FIG. 20 is a graphics user interface view of a simulation view showingtwo avatars in a moderately complex kitchen environment according to anembodiment of the invention.

FIG. 21 is a graphics user interface view of a simulation view showingtwo avatars in a reduced complexity kitchen environment according to anembodiment of the invention.

FIG. 22 is a flow chart showing the variation in simulation complexityresponsive to real-time monitoring of subject anxiety levels.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Turning to FIG. 1, the apparatus is denoted a reference number 1000 as awhole. Control module 1001 includes computer processing unit 1005 thatexecutes functions, procedures and routines embodied on non-transitorycomputer media. Control module 1001 is communicatively coupled tosimulation datastore 1010 which is a repository for computer-generatedfeatures in a computer-simulated environment. The computer-generatedfeatures may include background structures, landscapes, and staticviews. The computer-generated features may also include static andmoving objects to which the human subject 1050, immersed in the virtualcomputer-simulated environment engages with. Within the environment, thehuman subject 1050 is tasked to perform an executive function. Thisexecutive function requires a sufficient level of cognitive functionwhereby an increase in the anxiety state of the human subject 1050 wouldhinder the successful completion of the executive function.

It should be noted that the executive function may span a spectrum fromsimple object manipulation, public speaking to athletic performance. Forexample, an executive function could be a soccer player taking a penaltykick to determine the outcome of the World Cup. The human subject inthis example could be placed in a room having simulated turf, a ball,and a net. The computer simulation could be rendered to a head-mountedaugmented reality display (HMD) wherein the physically present ball canbe kicked but a capacity-filled soccer stadium is rendered in 360degrees onto the HMD worn by the human subject. The computer simulationrenders the spectators, opposing players and loud, ambient noise of thecheering stadium to acclimate the human subject to simulate the stressand anxiety of the situation. At some point, a sufficient level ofanxiety imparted on the human subject would hinder his or her ability tocompetently kick the ball into the net to score the penalty shot. In theparlance of sports, the human subject would “choke” and shoot poorly.

Returning back to FIG. 1, a rendering module 1015 is communicativelycoupled to the control module 1001. The rendering module is furthercommunicatively coupled to a graphic processing unit (GPU) 1020 thatgenerates the visual objects in the computer-simulated environment. Avisual display device 1025 is communicatively coupled to the renderingmodule 1015 and the GPU 1020. In the example above, it might be theaugmented reality HMD, such as those sold under the brand HOLOLENS byMicrosoft Corporation. Alternatively, the display device 1025 may besingle panel display monitors, groups of displays forming multi-paneldisplay monitors, rear projection displays, front projection displaysand virtual reality HMDs. These technologies provide different levels ofrealism and immersion into the computer-simulated environment.

Also coupled communicatively to the rendering module is an audioprocessing unit (APU) 1030, which generates audio output in thecomputer-simulated environment. This may include but is not limited to,background noise, background dialog, foreground dialog, and foregroundnoise. The APU 1030 may generate audio that is spatially coordinatedwith visual objects rendered through the GPU 1020. For example, anindiscernible background conversation forming ambient noise may bespatially aligned with a GPU 1020 rendered of two individuals off in adistance conversing.

An array of sensory variables 1035 is made accessible to the renderingmodule 1015. The sensory variables 1035 quantify an amount of visual andaudible information generated by the rendering module 1015 and presentedin the computer-simulated environment. As they are variables, they aresubject to modification in line with the objectives of the presentinvention. The values can be increased or decreased automatically. Thesesensory variables 1035 may include audible noise, audio volume, quantityof visual objects in the environment, movement of visual objects in theenvironment, polygon count of rendered objects in the environment,lighting complexity of the environment, texture complexity of renderedobjects in the environment, and rendered frames-per-second.

For example, in the case of audible noise, the variable could relate tothe frequency in which a bird chirps in the background. A low valuewould be an interval of once per minute. A relatively higher value wouldbe twenty (20) times per minute. In the case of quantity of visualobjects in the environment, a desk may be cluttered with pencils,staplers, tape, paper, computer peripherals and the like. For certainhuman subjects, a reduction of this clutter (by reducing the quantity ofvisual objects) reduces anxiety levels. For a computer-simulatedenvironment wherein the executive function task is to cross a busystreet, the movement of the vehicles (rendered objects) up and down thestreet may be automatically slowed or increased. Harsh or complexlighting may be imposed or reduced by adjusting the quantity of renderedlight sources, the light intensity, the diffusion of the shadows, andthe distance of the light source to objects. Texture complexity mayrelate to the realism of the computer-simulated environment. The texturecomplexity may be simplified or rendered in softer materials to mitigateanxious responses by the human subject. Frames per second rendered bythe display device may be used to mitigate or induce anxiety in somesubjects whereby high frame rates may be associated with more dynamicand action-based environments while slower frame rates may be consideredmore soothing.

A sensing module 1040 is provided and communicatively coupled to thecontrol module 1001. Coupled to the sensing module 1040 are at least oneor more digital sensors 1045 including cameras, radar, thermometers,heart rate monitor, pulse-oximeters, capacitive skin monitors andmicrophones. The sensing module 1040 receives non-transitory datasignals from the digital sensors 1045 indicative of a physiologicalparameter of the human subject 1050. This physiological parameter mayinclude but is not limited to, pulse rate, oxygen levels, bodytemperature, body movement body pose, speech patterns, speech volume,perspiration, and the like. A real-time anxiety value of the humansubject 1050 is quantified from the data signals indicative of thephysiological parameter wherein the real-time anxiety value is readableby the control module 1001.

An anxiety threshold datastore 1060 is communicatively coupled to thecontrol module 1001. The anxiety threshold datastore 1060 stores anupper anxiety state value constant 1065 representing a diminishedphysiological capability of performing executive functions. A loweranxiety state value constant 1070 is associated with a sufficiently lowphysiological anxiety state whereby executive functions may besuccessfully performed with additional stress-induced anxiety. The upperanxiety state value constant 1065 and the lower anxiety state valueconstant 1070 are computed by one or more quantitative factors selectedincluding, but not limited to, pulse rate, oxygen level, respirationrate, skin temperature and diaphoresis.

An anxiety threshold function 1075 operable on the control module 1001is provided. The anxiety threshold function 1075 receives a real-timeanxiety value of the human subject, the upper anxiety value constant,and the lower anxiety value constant. The anxiety threshold function1075 returns a low result responsive to the real-time anxiety value ofthe human subject being less than the lower anxiety value. The function1075 returns a high result responsive to the real-time anxiety value ofthe human subject being greater than the upper anxiety value. Finally,the function 1075 returns an inbounds result responsive to the real-timeanxiety value of the human subject being above the lower anxiety valueand less than the upper anxiety value.

Responsive to a low result returned from the anxiety threshold function1075, the control module 1001 instructs the rendering module 1015 toincrease the values of the sensory variables 1035 to thereby increasethe amount of visual and audible information generated by the renderingmodule 1015 and presented within the computer-simulated environment.Responsive to a high result returned from the anxiety threshold function1075, the control module 1001 instructs the rendering module 1015 todecrease the values of the sensory variables 1035 to thereby decreasethe amount of visual and audible information generated by the renderingmodule 1015 and presented within the computer-simulated environment.Finally, responsive to an inbounds result returned from the anxietythreshold function 1075, the control module 1001 instructs the renderingmodule 1015 to maintain substantially the same values of the sensoryvariables 1035 to thereby sustain the same amount of visual and audibleinformation generated by the rendering module 1015 and presented withinthe computer-simulated environment.

The human subject 1050 therapeutically develops proficiency in theexecutive function 1055 by optimizing the amount of visual and audibleinformation presented within the computer-simulated environment tosufficiently challenge the human subject 1050 by increasing visual andaudible information rendered in the computer-simulated environmentwithout detrimentally overloading the human subject 1050 with excessivevisual and audible information.

An exemplary anxiety threshold function 1075 is provided in FIG. 1 whichadjusts the sensory variables 1035 and the resultant complexity of thecomputer-simulated environment based on only pulse rate gathered fromthe digital sensors 1045 and passed through the sensing module 1040 tothe control module 1001.

A normal pulse for a healthy adult spans 60 to 100 beats per minute. Thepulse rate may vary and increase with anxiety. The exemplary anxietythreshold function in 1075 sets the upper anxiety value constant 1065with an integer value of 100. This represents 100 beats per minute. Thelower anxiety value constant 1070 is set at an integer value of 80. Thisrepresents 80 beats per minute. The value of 80 is set because 60-80 isstill considered normal and executive functions may be successfullyperformed with additional stress-induced anxiety up to at least 80. Ifonly a single threshold value was set, then the apparatus would“over-respond” to a movement above and below that single value.

In FIG. 2, a value of 105 beats per minute is received by the anxietythreshold function 1075. As the value of 105 exceeds the upper-valuethreshold 1065, the function 1075 returns a value of “decrease” to thecontrol module 1001. This instructs the control module (in this example)to decimate (reduce) the polygon count of the rendered computerenvironment by 20%. The rendering module 1015 then renders 20% lesspolygons and the simulation complexity is thereby reduced. The intendedeffect on the human subject 1050 is to reduce anxiety state and increasethe performance of the tasked executive functions 1055. This allows thetherapeutic session to continue without “breaking” the human subject byoverstimulation resulting in detrimental anxiety levels.

In FIG. 3, a value of 85 beats per minute is received by the anxietythreshold function 1075. As the value of 85 is between the upper-valuethreshold 1065 of 100 and the lower value threshold 1070 of 80, thefunction 1075 returns a value of “maintain” to the control module 1001.This instructs the control module (in this example) to maintain the samethe polygon count of the rendered computer environment. The renderingmodule 1015 renders the same number of polygons and the simulationcomplexity is thereby kept constant. The intended effect on the humansubject 1050 is to produce no change in the anxiety state and therebyextend the therapeutic session to continue to develop proficiency in thetasked executive functions 1055.

In FIG. 4, the human subject 1050 is not sufficiently challenged by theexecutive task and/or the computer-simulated environment. A value of 76beats per minute is received by the anxiety threshold function 1075. Asthe value of 76 is below the lower value threshold 1070 of 80, thefunction 1075 returns a value of “increase” to the control module 1001.This instructs the control module (in this example) to increase thepolygon count of the rendered computer environment (or revert to theoriginal polygon count). The rendering module 1015 renders an increasednumber of polygons and the simulation complexity is thereby increased.The intended effect on the human subject 1050 is to produce a morechallenging environment even at the possible cost of slowly increasinganxiety state. This maintains the therapeutic session efficacy in thetasked executive functions 1055 and mitigates the human subject 1050suffering from complacency or boredom.

FIG. 5 shows an alternative embodiment of the anxiety threshold function1075 using two physiological values (pulse and respiration) and alsomodifying two separate aspects of the computer-simulated environment. Inthis embodiment, human subject 1050 has a detected pulse of 95 and arespiration rate of 35 breaths per minute. Anxiety threshold function1075 receives both values. The normal respiration rate for an adult atrest is 12 to 20 breaths per minute. A respiration rate under 12 or over25 breaths per minute while resting is considered abnormal. The humansubject's pulse (at 95) is below the upper constant integer value of 100but below the lower constant integer value of 80. The human subject'srespiration of 35 breaths per minute is likewise well above normal andan indicium of anxiety. The function applied in FIG. 5 is unweightedcombining the values linearly, but it would be known to a person ofordinary skill in the art to weigh the physiological import ofrespiration versus pulse (or other measured factors as enumeratedabove). In this example, a value of “decrease” is returned to thecontrol module 1001, which reduces background sound volume of thecomputer simulation by 50 percent and also reduces the rendered framesper second from 30 to 15. The intended therapeutic effect on the humansubject 1050 is to reduce the anxiety level detected from abnormallyhigh breathing rate which will increase his or her capability ofsuccessfully performing the executive functions 1055.

Turning to FIG. 6, human subject 20 is engaged stacking rings on top ofeach other. A plurality of sensors monitors subject 20 including video30, audio 40, pulse rate 50, eye movement 60 and temperature 70.Different sensor apparatuses may be applied based on the needs of thetraining, maturity of the subject and clinician preferences. Forexample, a wrist-based band may measure temperature, pulse and bodymovements. A microphone may be parabolic or lapel. The video maysuperimpose thermal (infrared) images to show body temperature changes.

Sensor data from sensors 30, 40, 50, 60 and 70 are sent to sensor server80. The sensor data may be aggregated to form a total quantified anxietylevel. The sensor data may be weighted to each detected characteristic.For example, a normal heartrate is 60 to 100 beats per minute but mayincrease measurably due to anxiety. A resting heart rate for the subjectmay be obtained prior to the computer simulation and increases ordecreases from that starting rate may be weighted formulaically.

Sensor data related to body temperature may be associated withanxiety-based vasoconstriction, which may cause the body to heat up veryquickly. However, this may also be followed up by sweating, a naturalresponse to vasoconstriction which may cause body temperature drops.Accordingly, in some monitoring situations, body temperature data may becoupled with a perspiration sensor to detect changes of impedancewherein a cool-down from anxiety-induced temperature elevation fromvasoconstriction is not incorrectly associated with a reduction inanxiety but from sweating.

Clinical observations suggest abnormal gaze perception to be animportant indicator of anxiety disorders. In addition, vigilance inanxiety disorders may be conveyed by fixations on sources of stress.These behaviors and others related to them may be monitored byeye-tracking by camera sensors and weighted to anxiety levels. Speechpatterns may be linked to both diagnosis and immediate anxiety levelsbased on activation, tonality, and monotony among other characteristics.

Sensor data from sensor server 80 is relayed to simulation server 90.The sensor data may be aggregated into one, total numeric value, or maybe segmented into different subsets values for more granular control ofthe computer-simulated environment features.

FIG. 7 shows a graphic user interface 110 of a sensor view for patent130. Various controls on the interface include patient age 140,diagnosis 150, total session time 160, time left in the current session170, and current anxiety level 180. An end user operating the interfacemay select from an array 190 of sensors to individually select whichtypes of data to receive. Anxiety aggregate graph 200 shows a relativelysteady increase in subject 20 anxiety levels from six points in time(T1-T6). Simulation log 210 reports on incoming sensor data andadjustments automatically made in the computer simulation. Dialogcontrol 220 is a master complexity slider for the computer simulationset to automatic mode. The slider in dialog control 220 automaticallymoves left as the system reduces complexity responsive to elevatedsubject anxiety and moves right as the subject's anxiety leveldecreases. Selecting the manual radio button in dialog control 220permits the end user to override the overall complexity of thecomputer-simulated environment but the computer neverthelessintelligently and automatically reduces or increases multiple featuresof simulation complexity for the end user.

In FIG. 8, simulation view 230 is shown wherein audio control tab 250 isactivated. Radio button “Automatic” is selected in control tab 250 soboth ambient noises and avatar audio are automatically controlled by thecomputer. However, in this case, end user has manually “locked” oroverrode the automated control for the timbre of the avatar and thetimbre of the ambient noise in the simulation. Patient anxiety valuesfor multiple sensors are presented as a stacked graph 240 in theinterface view 230. Simulation log 210 shows that the computer,responsive to sensor-detected anxiety levels, decreased ambient noisevolume but increased avatar speech volume.

In FIG. 9, simulation environment tab 300 is active and shows simulationfeatures such as hue, saturation, lightness, number of objects andtexture complexity. In this example, the number of objects is locked atzero (0) which, in this embodiment, means that the default number ofobjects is retained (none added, and none removed). Texture in thesimulation has automatically been reduced. For some disorders,particularly those involving the processing of information, reducingtexture complexity of objects and backgrounds in a simulation may reducestress. For example, instead of a detailed, brick wall, the simulationmay show a plain painted surface. In some embodiments of the invention,objects may have movement loops such as fish in a fish tank or a plantmoving gently from a breeze. The movement of the object may be paused,slowed or speed up responsive to the sensed anxiety level of thesubject. For example, an object may even be a monitor in the backgroundof the simulated environment showing a newscast. Such movement andcontent may cause anxiety which is detected by the sensors, so thecontent of the monitor is changed to landscape images or the monitor isvirtually “turned off” to a black screen.

In FIG. 10, a different subject is entered into the interface diagnosedwith PTSD.Tab 310 shows a first avatar character. Settings for the firstavatar have a number of features including pose states, clothingoptions, polygon detail (e.g., realism). FIG. 11 shows a second avatarfor the same simulation and subject.

In FIG. 12, a simulation render dialog 330 is shown with a 2D simulationimage 360 of first avatar 370 in a kitchen environment. In FIG. 13, thesensor detects that the subject 20 has reduced anxiety levels so thecomplexity of the simulation lighting is increased to add more realism.FIG. 13 also shows a distance d₁ associated with the height of firstavatar 370 in the virtual environment. However, in FIG. 14, sensor dataconveyed an increase in subject 20 anxiety, so lighting complexity isreduced and optionally, the height of first avatar 370 is reduced 20% tod₂.

FIG. 15 shows an interface for a 12-year old patient diagnosed with Ringof Fire Attention Deficient Disorder in a middle school classroomenvironment. The simulation environment that subject 20 observes isshown in window 400 with second avatar 350. Simulation log notes thatthe schoolroom environment was intentionally initiated with alow-texture level and low polygon count to reduce initial stress on thisparticular subject. Second avatar 350 is rendered photorealistic. FIG.16 indicates an increase in anxiety for subject 20 so second avatar 350is simplified by removing the photorealistic UV mapping. Second avatar350 is conveyed in a revised simulation window 410 with less complexitywhich is indicated further by the far-left position of the slider indialog control 220.

FIG. 17 returns to the 7-year old subject diagnosed with Autism Level 1as previously shown in FIGS. 7-8. Patient view 420 is shown with a videofeed 430 of the patient interacting with blocks. Facial tracking points440 are overlaid on videofeed 430, which is translated into an anxietyvalue 450 along with body temperature 460, pulse 470 and speech volume480.

FIG. 18 returns to simulation view 230 with tactile tab 500 activeshowing tactile dialog 510. The computer automatically controls hapticresponse settings for intensity and frequency. The haptic response maybe a vibrational device affixed to the subject's limb, on a chair, undera play mat or within a toy being manipulated such as a stuffed animal.FIG. 19 shows olfactory tab 530 active displaying olfactory controldialog 540. It may control one or more scent diffusers that atomizemixtures of propylene glycol and scent elements.

The computer-simulated environment is shown in a 2D presentation inFIGS. 20 and 21. In FIG. 20, a computer interface is provided that showstwo avatars (Mike and Claire) in a kitchen setting. Data on the top ofthe interface show 92 objects in the scene. Ambient sound complexity(e.g., dishwasher, chirping birds, refrigerator compressor) is at level84. The maximum sound volume is 64 dB. The polygon count of thesimulation is 953,019, and the triangle count is 1.4 million. None ofthe non-avatar objects are moving. Lighting complexity is 75% andtextures on surfaces are 90% complexity. The aggregate anxiety level ofthe subject is 316 at 12 minutes and 42 seconds into a session todevelop executive functions in the context of cleaning a kitchen. Ascent diffuser is atomizing a solution conveying the smell of cookedchicken to the subject.

In FIG. 21, the computer makes modifications to the computer-simulationenvironment of FIG. 20 to reduce complexity and thus lower anxietylevels. Twenty-four objects are removed from the countertop includingplants, a paper towel roll, a toaster, a cutting board, a cup of spoons,and various other objects. Ambient sounds are reduced, and the maximumsound is at 45 decibels. The polygon and triangle counts are reduced,and the scent diffuser turned off. Lighting complexity is reduced andtextures such as those on the cabinets and counters are removed.

FIG. 22 shows a process according to an embodiment of the inventiondenoted as 610 as a whole. Initial observation 620 of subject 20 bysensors 630 derive a baseline anxiety level 650 from phenotypic anxietyattributes 640. The session begins and the system observes the tasksand/or training to develop executive functions 660. The phenotypicanxiety 640 is continually measured during the session. Increasinganxiety levels of subject 20 automatically causes computer simulation todecrease in complexity 690 by changing simulation features 700.Decreasing anxiety levels of subject 20 enables the computer simulationto reintroduce simulation complexity 680, which is intended to developthe subject's ability to perform executive functions in real-worldconditions.

Embodiments of the invention may be applied to numerous anxiety-intenseenvironments such as a computer-simulated environment of an aircraftcabin for the treatment of flight anxiety. The computer may activatemovement in a chair in which the subject sits to simulate airturbulence. The subject's pulse rapidly increases and his ability tocontinue through the therapeutic session is in jeopardy. The computerautomatically determines the subject needs a more calming environment.However, rather than simply terminate the turbulence simulation, theAI-driven avatar offers pre-recorded calming words to the subject thatthey are safe, and the ride will soon smooth out.

An embodiment of the present invention is a virtual, mixed, and/oraugmented reality environment for the patient. Several key componentsare adaptable under the invention:

Environment: For example, the experience can take place in a home,classroom, work or outdoor environment.

Display: The experiences can be delivered on a wide variety of displaytypes. These currently include laptop, large screen TV, full wallprojection, and full surround as enabled by a CAVE (Cave AutomaticVirtual Environment), a VR (Virtual Reality), an AR (Augmented Reality)or MR (Mixed Reality) headset.

Virtual or Mixed: The experience can be purely virtual or can blendvirtual objects with the physical setting. Purely virtual experiencesare common, but we can include the real setting as part of theexperience. Clearly, if a headset is used, the blending of the real andvirtual can be done with a variety of existing and evolving technologiesand algorithms (e.g., HoloLens or Magic Leap or Vive Pro).

Number of Avatars in Environment: The virtual environment can have oneor even many avatars.

Diversity of Avatars: The avatars that populate the environment can beof varying ages, ethnicities, and behavioral/cultural/family situationprofiles.

Ability to Capture, Analyze and Adapt: The system includes capabilitiesto capture “performances” and to support annotations by subject-matterexperts. The system automates much of the analysis and adds the abilityto adapt the experience in real-time, based on these analyses.

Annotations describing a participant's affective states can be used inreflective learning. The invention supports this activity by focusing oncapturing, analyzing and identifying nonverbal cues during cyberlearning experiences. These cues can then be offered to subject-matterexperts in support of their providing annotations (semi-automated) orcan be used to directly specify annotations without human intervention(automated). Challenges that one encounters here in both body gesturesand facial expressions include self-occlusion. A particular example ofthis in facial expressions is hand-to-face occlusion (common withchildren with ASD). The present invention includes synthesized versionsof these occlusions that are used to train a deep learning system toidentify expressions in the context of hidden landmarks.

In addition to dealing with self-occlusion, the present inventionincludes a novel machine learning approach to explicitly disentanglefacial expression representations from identity-related information. Thedisentangled facial expression representation can then be used toidentify emotional responses or even to impose these same emotionalresponses on avatars, using the unique characteristics of each avatar.

GLOSSARY OF CLAIM TERMS

Anxiety means a nervous condition characterized by a state of excessiveuneasiness and apprehension.

Augmented Reality means technologies that superimpose acomputer-generated image on a user's view of the real world, thusproviding a composite view.

Autism Spectrum Disorder (ASD) is a developmental disorder that affectscommunication and behavior. Autism is known as a “spectrum” disorderbecause there is wide variation in the type and severity of symptomspeople experience.

Avatar means an icon or figure representing a particular character in acomputer simulation. For the purposes of this specification, the avataris an automated, computer-controlled object rendered it thecomputer-simulated environment representing a human or other animals(e.g., canine).

Baseline means the starting anxiety level prior to, or at the time thecomputer simulation is initiated. Baseline levels may be obtained undermoderate to low environmental stimulation but should be consistentlymeasured under the same conditions between therapeutic sessions.

Executive functioning (EF) means brain processes that include (but arenot necessarily limited to) inhibition, memory, attention, flexibility,planning, and problem-solving.

Haptic means perception and manipulation of objects using the senses oftouch and proprioception.

Head Mounted Display (HMD) is a digital display device worn on the heador integrated into a helmet. A HMD may present a completely virtualreality environment or may also reflect projected images wherein a usermay see through it in augmented reality environments. Some commerciallyavailable HMDs include those sold under the brands' OCULUS RIFT andMICROSOFT HOLOLENS.

Mixed Reality means the combination of virtual and real worlds togenerate new environments and visualizations wherein physical anddigital objects co-exist and interact in real-time.

Olfactory means relating to the sense of smell.

Phenotypic anxiety level means observable displays of anxiety detectableby quantified, sensor-implemented monitoring of an individual.

Sensor means a device that detects or measures a physical property andrecords or conveys its value. In the case of the present invention, asensor monitors visual, audio, temperature, and other physicalproperties of a human subject.

Sensory overload is when one or more of the body's senses experiences inan individual experiences and overload that causes stress or anxiety inthe person.

Tactile means of or connected with the sense of touch.

UV mapping means the 3D modeling process of projecting a 2D image to a3D model's surface for texture mapping.

Virtual Environment means the audio, visual, tactile, and other sensoryfeatures of a computer-generated simulation.

Virtual Reality means a computer-generated simulation of athree-dimensional image or environment that can be interacted with in aseemingly real or physical way by a person using special electronicequipment, such as a helmet with a screen inside or gloves fitted withsensors

The advantages set forth above, and those made apparent from theforegoing description, are efficiently attained. Since certain changesmay be made in the above construction without departing from the scopeof the invention, it is intended that all matters contained in theforegoing description or shown in the accompanying drawings shall beinterpreted as illustrative and not in a limiting sense.

What is claimed is:
 1. An apparatus comprising: a control modulecomprising a computer processor communicatively coupled to a simulationdata store, the simulation data store having machine-readable values forcomputer-generated features in a computer-simulated environment in whicha human subject is immersed, the computer-generated features includevisual objects; a rendering module communicatively coupled to thecontrol module; the rendering module generates the visual objects in thecomputer-simulated environment; a visual display device communicativelycoupled to the rendering module, the visual display device displayingthe visual objects in the computer-simulated environment; an array ofsensory variables accessible by the rendering module, the sensoryvariables quantifying an amount of visual information generated by therendering module and presented in the computer- simulated environment,the sensory variables selected from the group consisting of polygoncount of rendered objects in the environment, lighting complexity of theenvironment, texture complexity of rendered objects in the environmentand rendered frames-per-second; a sensing module communicatively coupledto the control module and at least one or more digital sensors, thesensing module receiving data from the digital sensors indicative of aphysiological parameter of the human subject and quantifying a real-timesensory load value of the human subject from the data indicative of thephysiological parameter, the real-time sensory load value readable bythe control module; a sensory load threshold datastore communicativelycoupled to the control module, the sensory load threshold datastorestoring an upper sensory load state value constant, the sensory loadthreshold datastore also storing a lower sensory load state valueconstant; a sensory load threshold function operable on the controlmodule, the sensory load threshold function receiving the real-timesensory load value of the human subject, the upper sensory load valueconstant and the lower sensory load value constant whereby the sensoryload threshold function returns a low result responsive to the real-timesensory load value of the human subject being less than the lowersensory load value; a high result responsive to the real-time sensoryload value of the human subject being greater than the upper sensoryload value; and an inbounds result responsive to the real-time sensoryload value of the human subject being above the lower sensory load valueand less than the upper sensory load value; whereby responsive to a lowresult returned from the sensory load threshold function, the controlmodule instructs the rendering module to increase the values of thesensory variables to thereby increase the amount of visual informationgenerated by the rendering module and presented within thecomputer-simulated environment; responsive to a high result returnedfrom the sensory load threshold function, the control module instructsthe rendering module to decrease the values of the sensory variables tothereby decrease the amount of visual information generated by therendering module and presented within the computer-simulatedenvironment; and responsive to an inbounds result returned from thesensory load threshold function, the control module instructs therendering module to maintain substantially the same values of thesensory variables to thereby sustain the same amount of visualinformation generated by the rendering module and presented within thecomputer-simulated environment
 2. The apparatus of claim 1 wherein thephysiological parameter detectable by the at least one or more sensorsis selected from the group consisting of facial tracking, body movement,body temperature, pulse rate, respiratory rate, eye movement, and speechpatterns.
 3. The apparatus of claim 1 wherein the visual display deviceis selected from the group consisting of single panel display monitors,multi-panel display monitors, rear projection displays, front projectiondisplays, head-mounted virtual reality displays, and head-mountedaugmented reality displays.
 4. The apparatus of claim 1 furthercomprising an audio processing unit (APU)communicatively coupled to therendering module, the APU generates an audio output in thecomputer-simulated environment
 5. The apparatus of claim 1 wherein thedigital sensors are selected from the group consisting of cameras,radar, thermometers, heart rate monitor, pulse-oximeters, andmicrophones.
 6. The apparatus of claim 1 wherein the rendering modulegenerates a computer-generated simulation selected from the groupconsisting of a classroom, a workplace, a vehicle, a battlefield, ahospital and an athletic event
 7. An apparatus comprising: a controlmodule comprising a computer processor communicatively coupled to asimulation data store, the simulation data store having machine-readablevalues for computer-generated features in a computer-simulatedenvironment in which a human subject is immersed, the computer-generatedfeatures include an audio output; a rendering module communicativelycoupled to the control module; the rendering module generates the audiooutput in the computer-simulated environment; an audio output devicecommunicatively coupled to the rendering module, the audio output devicebroadcasting the audio output in the computer-simulated environment; anarray of audio track variables accessible by the rendering module, theaudio track variables quantifying an amount of audio informationgenerated by the rendering module and presented in thecomputer-simulated environment; a sensing module communicatively coupledto the control module and at least one or more digital sensors, thesensing module receiving data from the digital sensors indicative of aphysiological parameter of the human subject and quantifying a real-timesensory load value of the human subject from the data indicative of thephysiological parameter, the real-time sensory load value readable bythe control module; a sensory load threshold datastore communicativelycoupled to the control module, the sensory load threshold datastorestoring an upper sensory load state value constant, the sensory loadthreshold datastore also storing a lower sensory load state valueconstant; a sensory load threshold function operable on the controlmodule, the sensory load threshold function receiving the real-timesensory load value of the human subject, the upper sensory load valueconstant and the lower sensory load value constant whereby the sensoryload threshold function returns a low result responsive to the real-timesensory load value of the human subject being less than the lowersensory load value; a high result responsive to the real-time sensoryload value of the human subject being greater than the upper sensoryload value; and an inbounds result responsive to the real-time sensoryload value of the human subject being above the lower sensory load valueand less than the upper sensory load value; whereby responsive to a lowresult returned from the sensory load threshold function, the controlmodule instructs the rendering module to increase the values of theaudio track variables to thereby increase the amount of audibleinformation generated by the rendering module and presented within thecomputer-simulated environment; responsive to a high result returnedfrom the sensory load threshold function, the control module instructsthe rendering module to decrease the values of the sensory variables tothereby decrease the amount of audible information generated by therendering module and presented within the computer-simulatedenvironment; and responsive to an inbounds result returned from thesensory load threshold function, the control module instructs therendering module to maintain substantially the same values of thesensory variables to thereby sustain the same amount of audibleinformation generated by the rendering module and presented within thecomputer-simulated environment.
 8. The apparatus of claim 7 wherein theaudio track variables are selected from the group consisting ofbackground noise, background dialog, foreground dialog and foregroundnoise.
 9. The apparatus of claim 7 wherein the audio track variablesinclude broadcast interval.
 10. The apparatus of claim 7 wherein thephysiological parameter detectable by the at least one or more sensorsis selected from the group consisting of facial tracking, body movement,body temperature, pulse rate, respiratory rate, eye movement, and speechpatterns.
 11. The apparatus of claim 7 further comprising an audioprocessing unit (APU) communicatively coupled to the rendering module,the APU generates an audio output in the computer-simulated environment12. The apparatus of claim 7 wherein the digital sensors are selectedfrom the group consisting of cameras, radar, thermometers, heart ratemonitor, pulse-oximeters, and microphones.
 13. An apparatus comprising:a control module comprising a computer processor communicatively coupledto a simulation data store, the simulation data store havingmachine-readable values for computer-generated features in acomputer-simulated environment in which a human subject is immersed, thecomputer-generated features include visual objects and an audio output;a rendering module communicatively coupled to the control module; therendering module generates the visual objects in the computer- simulatedenvironment; a visual display device communicatively coupled to therendering module, the visual display device displaying the visualobjects in the computer-simulated environment; an array of sensoryvariables accessible by the rendering module, the sensory variablesquantifying an amount of visual information generated by the renderingmodule and presented in the computer- simulated environment, the sensoryvariables selected from the group consisting of polygon count ofrendered objects in the environment, lighting complexity of theenvironment, texture complexity of rendered objects in the environmentand rendered frames-per-second; a sensing module communicatively coupledto the control module and at least one or more digital sensors, thesensing module receiving data from the digital sensors indicative of thehuman subject's pulse rate and quantifying a real-time sensory loadvalue of the human subject from the pulse rate, the real-time sensoryload value readable by the control module; a sensory load thresholddatastore communicatively coupled to the control module, the sensoryload threshold datastore storing an upper sensory load state valueconstant, the sensory load threshold datastore also storing a lowersensory load state value constant; a sensory load threshold functionoperable on the control module, the sensory load threshold functionreceiving the real-time sensory load value of the human subject, theupper sensory load value constant and the lower sensory load valueconstant whereby the sensory load threshold function returns a lowresult responsive to the real-time sensory load value of the humansubject being less than the lower sensory load value; a high resultresponsive to the real-time sensory load value of the human subjectbeing greater than the upper sensory load value; and an inbounds resultresponsive to the real- time sensory load value of the human subjectbeing above the lower sensory load value and less than the upper sensoryload value; whereby responsive to a low result returned from the sensoryload threshold function, the control module instructs the renderingmodule to increase the values of the sensory variables to therebyincrease the amount of visual information generated by the renderingmodule and presented within the computer-simulated environment;responsive to a high result returned from the sensory load thresholdfunction, the control module instructs the rendering module to decreasethe values of the sensory variables to thereby decrease the amount ofvisual information generated by the rendering module and presentedwithin the computer-simulated environment; and responsive to an inboundsresult returned from the sensory load threshold function, the controlmodule instructs the rendering module to maintain substantially the samevalues of the sensory variables to thereby sustain the same amount ofvisual information generated by the rendering module and presentedwithin the computer-simulated environment.
 14. The apparatus of claim 13wherein the visual display device is selected from the group consistingof single panel display monitors, multi-panel display monitors, rearprojection displays, front projection displays, head-mounted virtualreality displays, and head-mounted augmented reality displays.
 15. Theapparatus of claim 13 further comprising an audio processing unit (APU)communicatively coupled to the rendering module, the APU generates anaudio output in the computer-simulated environment.
 16. The apparatus ofclaim 13 wherein the digital sensors are selected from the groupconsisting of cameras, radar, thermometers, heart rate monitor,pulse-oximeters, and microphones.